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ABSTRACT 

Researchers spend a great deal of time reading research pa¬ 
pers. However, this skill is rarely taught, leading to much 
wasted effort. This article outlines a practical and efficient 
three-pass method for reading research papers. I also de¬ 
scribe how to use this method to do a literature survey. 
Categories and Subject Descriptors: A.l [Introductory 
and Survey] 

General Terms: Documentation. 

Keywords: Paper, Reading, Hints. 

1. INTRODUCTION 

Researchers must read papers for several reasons: to re¬ 
view them for a conference or a class, to keep current in 
their field, or for a literature survey of a new field. A typi¬ 
cal researcher will likely spend hundreds of hours every year 
reading papers. 

Learning to efficiently read a paper is a critical but rarely 
taught skill. Beginning graduate students, therefore, must 
learn on their own using trial and error. Students waste 
much effort in the process and are frequently driven to frus¬ 
tration. 

For many years I have used a simple approach to efficiently 
read papers. This paper describes the ‘three-pass’ approach 
and its use in doing a literature survey. 

2. THE THREE-PASS APPROACH 

The key idea is that you should read the paper in up to 
three passes, instead of starting at the beginning and plow¬ 
ing your way to the end. Each pass accomplishes specific 
goals and builds upon the previous pass: The first pass 
gives you a general idea about the paper. The second pass 
lets you grasp the paper’s content, but not its details. The 
third pass helps you understand the paper in depth. 

2.1 The first pass 

The first pass is a quick scan to get a bird’s-eye view of 
the paper. You can also decide whether you need to do any 
more passes. This pass should take about five to ten minutes 
and consists of the following steps: 

1. Carefully read the title, abstract, and introduction 

2. Read the section and sub-section headings, but ignore 
everything else 

3. Glance at the mathematical content (if any) to deter¬ 
mine the underlying theoretical foundations 


4. Read the conclusions 

5. Glance over the references, mentally ticking off the 
ones you’ve already read 

At the end of the first pass, you should be able to answer 
the five Cs: 

1. Category : What type of paper is this? A measure¬ 
ment paper? An analysis of an existing system? A 
description of a research prototype? 

2. Context : Which other papers is it related to? Which 
theoretical bases were used to analyze the problem? 

3. Correctness : Do the assumptions appear to be valid? 

4. Contributions: What are the paper’s main contribu¬ 
tions? 

5. Clarity: Is the paper well written? 

Using this information, you may choose not to read fur¬ 
ther (and not print it out, thus saving trees). This could be 
because the paper doesn’t interest you, or you don’t know 
enough about the area to understand the paper, or that the 
authors make invalid assumptions. The first pass is ade¬ 
quate for papers that aren’t in your research area, but may 
someday prove relevant. 

Incidentally, when you write a paper, you can expect most 
reviewers (and readers) to make only one pass over it. Take 
care to choose coherent section and sub-section titles and 
to write concise and comprehensive abstracts. If a reviewer 
cannot understand the gist after one pass, the paper will 
likely be rejected; if a reader cannot understand the high¬ 
lights of the paper after five minutes, the paper will likely 
never be read. 

2.2 The second pass 

In the second pass, read the paper with greater care, but 
ignore details such as proofs. It helps to jot down the key 
points, or to make comments in the margins, as you read. 
In particular, Dominik Grusemann from Uni Augsburg sug¬ 
gests that “you may want to note down terms you didn’t 
understand, or questions you may want to ask the author.” 

1. Look carefully at the figures, diagrams and other illus¬ 
trations in the paper. Pay special attention to graphs. 
Are the axes properly labeled? Are results shown with 
error bars, so that conclusions are statistically sig¬ 
nificant? Common mistakes like these will separate 
rushed, shoddy work from the truly excellent. 



2. Remember to mark relevant unread references for fur¬ 
ther reading (this is a good way to learn more about 
the background of the paper). 

The second pass should take up to an hour. After this 
pass, you should be able to grasp the content of the paper. 
You should be able to summarize the main thrust of the pa¬ 
per, with supporting evidence, to someone else. This level of 
detail is appropriate for a paper in which you are interested, 
but does not lie in your research speciality. 

Sometimes you won’t understand a paper even at the end 
of the second pass. This may be because the subject matter 
is new to you, with unfamiliar terminology and acronyms. 
Or the authors may use a proof or experimental technique 
that you don’t understand, so that the bulk of the pa¬ 
per is incomprehensible. The paper may be poorly written 
with unsubstantiated assertions and numerous forward ref¬ 
erences. Or it could just be that it’s late at night and you’re 
tired. You can now choose to: (a) set the paper aside, hoping 
you don’t need to understand the material to be successful 
in your career, (b) return to the paper later, perhaps after 
reading background material or (c) persevere and go on to 
the third pass. 

2.3 The third pass 

To fully understand a paper, particularly if you are re¬ 
viewer, requires a third pass. The key to the third pass 
is to attempt to virtually re-implement the paper: that is, 
making the same assumptions as the authors, re-create the 
work. By comparing this re-creation with the actual paper, 
you can easily identify not only a paper’s innovations, but 
also its hidden failings and assumptions. 

This pass requires great attention to detail. You should 
identify and challenge every assumption in every statement. 
Moreover, you should think about how you yourself would 
present a particular idea. This comparison of the actual 
with the virtual lends a sharp insight into the proof and 
presentation techniques in the paper and you can very likely 
add this to your repertoire of tools. During this pass, you 
should also jot down ideas for future work. 

This pass can take about four or five hours for beginners, 
and about an hour for an experienced reader. At the end 
of this pass, you should be able to reconstruct the entire 
structure of the paper from memory, as well as be able to 
identify its strong and weak points. In particular, you should 
be able to pinpoint implicit assumptions, missing citations 
to relevant work, and potential issues with experimental or 
analytical techniques. 

2.4 Timing between passes 

Andrew Simpson from Queen Mary University of London 
writes: “the process works best when there is quite a large 
amount of time between each stage of review. For example, 
I find that I often collect large numbers of papers, read them 
(10 minute rapid first review) and then come back to them 
sometimes weeks later for a second pass. Finally, some fur¬ 
ther weeks or months later I return to them and find that 
I can extract the final useful insight that I had not done 
previously.” 

I agree with this approach, though it is not quite useful 
when facing a paper reviewing deadline! 

3. DOING A LITERATURE SURVEY 


Paper reading skills are put to the test in doing a literature 
survey. This will require you to read tens of papers, perhaps 
in an unfamiliar field. What papers should you read? Here 
is how you can use the three-pass approach to help. 

First, use an academic search engine such as Google Scholar 
or CiteSeer and some well-chosen keywords to find three to 
five recent papers in the area. Do one pass on each pa¬ 
per to get a sense of the work, then read their related work 
sections. You will find a thumbnail summary of the recent 
work, and perhaps, if you are lucky, a pointer to a recent 
survey paper. If you can find such a survey, you are done. 
Read the survey, congratulating yourself on your good luck. 

Otherwise, in the second step, find shared citations and 
repeated author names in the bibliography. These are the 
key papers and researchers in that area. Download the key 
papers and set them aside. Then go to the websites of the 
key researchers and see where they’ve published recently. 
That will help you identify the top conferences in that field 
because the best researchers usually publish in the top con¬ 
ferences. 

The third step is to go to the website for these top con¬ 
ferences and look through their recent proceedings. A quick 
scan will usually identify recent high-quality related work. 
These papers, along with the ones you set aside earlier, con¬ 
stitute the first version of your survey. Make two passes 
through these papers. If they all cite a key paper that you 
did not find earlier, obtain and read it, iterating as neces¬ 
sary. 

4. BENEFITS 

I’ve used this approach for the last 22 years to read con¬ 
ference proceedings, write reviews, do background research, 
and to quickly review papers before a discussion. This dis¬ 
ciplined approach prevents me from drowning in the details 
before getting a bird’s-eye-view. It allows me to estimate the 
amount of time required to review a set of papers. More¬ 
over, I can adjust the depth of paper evaluation depending 
on my needs and how much time I have. 

5. RELATED WORK 

If you are reading a paper to do a review, you should also 
read Timothy Roscoe’s paper on “Writing reviews for sys¬ 
tems conferences” [3]. If you’re planning to write a technical 
paper, you should refer both to Henning Schulzrinne’s com¬ 
prehensive web site [4] and George Whitesides’s excellent 
overview of the process [5]. Finally, Simon Peyton Jones 
has a website that covers the entire spectrum of research 
skills [2]. 

Iain H. McLean of Psychology, Inc. has put together a 
downloadable ‘review matrix’ that simplifies paper review¬ 
ing using the three-pass approach for papers in experimen¬ 
tal psychology[1], which can probably be used, with minor 
modifications, for papers in other areas. 

6. A REQUEST 

I would like to make this a living document, updating it 
as I receive comments. Please take a moment to email me 
any comments or suggestions for improvement. 
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